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Highlights 


Why  GAO  Did  This  Study 


What  GAO  Found 


Over  the  past  two  and  a  half  decades,  the 
Department  of  Defense  (DOD)  has 
invested  heavily  to  acquire  a  cruise 
missile  capable  of  attacking  groimd 
targets  stealthily,  reliably,  and  affordably. 
After  abandoning  an  earlier,  more 
expensive  missile  and  ajoint  service 
effort,  the  Air  Force  began  producing  the 
Joint  Air-to-Surface  Standoff  Missile 
(JASSM)  in  2001.  After  that,  the  program 

(1)  encoimtered  many  flight  test  failures, 

(2)  decided  to  develop  an  extended  range 
version,  and  (3)  recognized  significant 
cost  growth.  The  production  decision  for 
the  JASSM-ER  is  planned  for  November 
2010.  Also,  the  Secretary  of  Defense  has 
recently  announced  a  major  initiative  to 
restore  affordability  and  productivity  in 
defense  spending.  This  initiative  is 
expected  to,  among  other  things,  identify 
savings  by  conducting  needed  programs 
more  efficiently. 

As  DOD  faces  the  initial  production 
decision  on  JASSM-ER,  GAO  was  asked 
to  assess  (1)  most  recent  test  results, 
correction  of  causes  of  previous  flight 
test  failures,  and  efforts  to  improve 
JASSM’s  reliability;  and  (2)  JASSM  cost 
changes,  efforts  to  control  costs,  and 
additional  cost  risks  for  the  program. 

What  GAO  Recommends 

GAO  recommends  that  the  Secretary  of 
Defense  reevaluate  the  JASSM  program’s 
affordability  and  cost-effectiveness 
before  making  the  decision  to  produce 
the  JASSM-ER.  DOD  partially  concurred 
with  GAO’s  assessment,  but  believes  the 
JASSM-ER  should  begin  production  in 
November  2010.  GAO  believes  that  it  is 
incumbent  upon  the  department  to 
reexamine  JASSM  before  making  the 
production  decision  to  ensure  that  the 
program  is  structured  as  efficiently  as 
possible  and  is  still  a  good  investment 
given  the  other  demands  DOD  faces. 


Since  2007,  design  changes  and  other  corrective  actions  by  the  Air  Force  have 
improved  the  baseline  JASSM’s  test  results  significantly — the  missile  has  now 
demonstrated  85  percent  success  versus  58  percent  achieved  previously  and 
before  the  corrections.  The  JASSM-ER  variant  has  done  well  thus  far,  with  no 
failures  during  the  first  seven  flight  tests.  These  results  reflect  the  Air  Force’s 
enhanced  oversight  of  the  program  and  significant  investments  made  to  improve 
reliability.  These  efforts  also  identified  many  of  the  root  causes  for  flight  test 
failures.  While  baseline  JASSM  missile  reliability  has  improved,  it  is  not  expected 
to  achieve  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics’  required  level  of  90  percent  until  2013.  Tests  conducted  thus  far  of  the 
improved  baseline  JASSM  and  the  JASSM-ER  variants  have  been  developmental — 
or  controlled — in  nature.  Neither  the  improved  JASSM  baseline  missile  nor  the 
JASSM-ER  has  been  demonstrated  in  operationally  realistic  testing  or  in  a  combat 
operation. 


JASSM  Estimated  Costs  and  Quantities  Through  2025 

Dec.  1998 

May  2008 

Dec.  2009 

Quantities  to  be  produced 

2,400 

4,900 

4,900 

Total  Program  Cost  (millions) 

$2,232.5 

$6,641.4 

$7,129.4 

Program  Unit  Cost  (thousands) 

$907.0 

$1,327.0 

$1,421.0 

Procurement  Unit  Cost  (thousands) 

$515.0 

$1,073.0 

$1,160.0 

Source:  GAO  analysis  of  Department  of  Defense  data. 


Note:  Fiscal  year  2010  dollars. 


JASSM  costs  have  increased  by  over  seven  percent  since  the  program  was 
restructured  in  2008.  As  the  table  shows,  since  1998,  JASSM  quantities  have  more 
than  doubled  and  estimated  program  costs  have  grown  from  $2.2  billion  to  a  $7.1 
billion.  The  Air  Force  has  taken  several  steps  to  control  JASSM  costs,  but  options 
to  reduce  costs  at  this  point  appear  limited.  In  fact,  several  factors  suggest 
additional  cost  growth  is  likely.  First,  the  Air  Force  has  not  been  able  to  provide 
enough  funding  to  produce  the  missiles  at  planned  rates.  That  has  led  to  a  less 
efficient  production  process,  a  longer  production  period,  and  higher  costs  that 
have  not  yet  been  reflected  in  the  $7.1  billion  estimate.  Second,  the  Air  Force’s 
potential  plans  to  retrofit  existing  missiles  with  the  reliability  improvements  may 
not  be  feasible,  given  the  missile’s  sensitivity  to  being  reopened.  If  retrofits  prove 
infeasible,  new  replacements  may  have  to  be  purchased;  if  they  are  feasible,  the 
Air  Force  may  have  to  provide  additional  funding  to  retrofit  all  existing  missiles. 
Finally,  since  the  Air  Force  last  compared  JASSM  to  possible  alternatives,  the  unit 
cost  was  assumed  to  be  about  40  percent  less  than  currently  expected  and  that 
now  could  make  alternatives  more  competitive  in  terms  of  cost  and/or 
capabilities.  A  reevaluation  of  the  JASSM  program,  given  that  most  of  its  costs 
have  yet  to  be  incurred,  is  warranted  before  the  decision  to  produce  the  JASSM- 
ER  is  made. 
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GAO 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


October  13,  2010 
Congressional  Committees 

Over  the  past  two  and  a  half  decades,  the  Department  of  Defense  (DOD) 
has  invested  heavily  to  acquire  an  Air  Force  air-to-ground  cruise  missile 
capable  of  stealthy  flight  and  reliable  performance  at  affordable  costs.  The 
Air  Force  invested  in  the  Tri-Service  Standoff  Attack  Missile  (TSSAM) 
from  1986  through  1994  but  ended  the  program  once  the  unit  costs 
exceeded  $2  million  per  missile  and  as  testing  issues  surfaced.  The  Air 
Force  is  currently  in  production  for  the  Joint  Air-to-Surface  Standoff 
Missile  (JASSM).  This  program  is  intended  to  provide  a  next-generation 
cruise  missile  capable  of  striking  high-value,  highly  defended  targets  from 
outside  the  enemy’s  air  defenses  launched  from  a  variety  of  aircraft, 
including  the  B-l,  B-2,  B-52,  and  F-16.  Over  the  first  several  years  of  the 
program,  the  Air  Force  (1)  encountered  many  flight  test  failures,  (2) 
decided  to  develop  an  extended  range  version,  and  (3)  recognized 
significant  cost  growth  which  led  to  a  critical  Nunn-McCurdy  unit  cost 
breach  in  2006. 1  The  Air  Force  is  about  to  begin  producing  the  extended 
range  version  (JASSM-ER)  that  will  more  than  double  the  range  of  the 
baseline  missile.  The  two  variants  of  JASSM  are  approximately  70  percent 
common  in  hardware  and  95  percent  common  in  software. 

As  DOD  approaches  the  production  decision  on  JASSM-ER,  you  asked  us 
(1)  to  assess  the  results  of  testing,  whether  the  causes  of  previous  flight 
test  failures  have  been  corrected,  and  Air  Force  initiatives  to  improve 


1 10  U.S.C.  §  2433  establishes  the  requirement  for  DOD  to  prepare  unit  cost  reports  on 
major  defense  acquisition  programs  or  designated  major  defense  subprograms.  If  a 
program  exceeds  cost  growth  thresholds  specified  in  the  law,  this  is  known  as  a  Nuiin- 
McCurdy  breach.  The  law  describes  two  types  of  cost  growth  thresholds  that  can  be 
breached;  the  “significant  cost  growth  threshold”  and  the  “critical  cost  growth  threshold.” 
A  breach  of  the  critical  cost  growth  threshold  occurs  when  a  major  defense  acquisition 
program  or  designated  major  defense  subprogram  experiences  at  least  a  25  percent 
increase  over  either  the  program  acquisition  unit  cost  (total  cost  of  development, 
procurement,  and  system-specific  military  construction  divided  by  the  number  of  fully 
configured  end  items  to  be  procured)  or  over  the  procurement  unit  cost  (total  funds 
programmed  for  procurement  divided  by  the  total  number  of  fully  configured  end  items  to 
be  procured)  in  the  current  baseline  estimate  or  at  least  a  50  percent  increase  over  the 
program  acquisition  unit  cost  or  the  procurement  unit  cost  in  the  original  baseline 
estimate.  DOD  is  also  required  to  report  breaches  to  Congress.  At  the  time  of  JASSM’s 
breach,  section  2433  also  required  DOD  to,  in  certain  circumstances,  reassess  the  program 
and  submit  a  certification  to  Congress  in  order  to  continue  the  program.  Currently,  these 
and  other  requirements  are  found  in  10  U.S.C.  §  2433a. 
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JASSM’s  reliability;  and  (2)  to  identify  the  extent  JASSM  costs  have 
changed  over  time,  the  steps  the  Air  Force  is  taking  to  control  costs,  and 
additional  cost  risks,  if  any,  for  the  program.  We  note  that  the  production 
decision  on  JASSM-ER  is  being  made  shortly  after  the  Secretary  of 
Defense  announced  a  major  initiative  to  improve  the  cost  efficiency  of 
weapon  systems  acquisition. 

To  determine  JASSM’s  current  costs  and  the  extent  they  have  changed,  we 
analyzed  JASSM’s  contracts,  budgets,  and  selected  acquisition  reports.  We 
compared  the  cost  estimating  practices  of  Office  of  Secretary  of  Defense 
(OSD)  cost  analysts  in  the  development  of  the  life-cycle  cost  estimates  for 
JASSM  and  JASSM-ER  to  our  Cost  Estimating  and  Assessment  Guide.* 2  To 
assess  the  results  of  the  most  recent  tests  and  to  determine  if  corrective 
actions  have  been  implemented,  we  analyzed  JASSM  flight  test  results  and 
the  results  of  the  failure  review  boards  to  determine  scoring  criteria  and 
results  and  what  corrective  actions  were  implemented.  We  interviewed 
officials  with  the  JASSM  joint  program  office,  Lockheed  Martin — the 
prime  contractor,  the  Air  Force,  and  Office  of  Secretary  of  Defense.  We 
interviewed  a  former  Air  Force  official  who  was  an  early  JASSM  program 
manager  to  better  understand  the  program’s  objectives  and  original 
acquisition  strategy.  We  also  reviewed  our  prior  work  on  best  acquisition 
practices  which  established  a  set  of  evaluation  criteria  for  a  knowledge- 
based  acquisition  approach.3  We  conducted  this  performance  audit  from 
November  2009  through  October  2010  in  accordance  with  generally 
accepted  government  auditing  standards.  Those  standards  require  that  we 
plan  and  perform  the  audit  to  obtain  sufficient,  appropriate  evidence  to 
provide  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our 
audit  objectives.  We  believe  that  the  evidence  obtained  provides  a 
reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit 
objectives. 


Background 


The  JASSM  program  began  in  1995  and  was  to  be  an  affordable,  joint 
program  between  the  Air  Force  and  the  Navy  to  meet  an  urgent  need  with 
a  streamlined  acquisition  strategy.  JASSM  predecessor  TSSAM  was  also 
planned  to  be  a  low-cost  cruise  missile  able  to  deliver  several  different 


2  GAO,  GAO  Cost  Estimating  and  Assessment  Guide,  GAO-09-3SP  (Washington,  D.C.: 
March  2009). 

3  GAO,  Defense  Acquisitions:  Strong  Leadership  is  Key  to  Planning  and  Executing  Stable 
Weapon  Programs,  GAO-10-522  (Washington,  D.C.:  May  6,  2010). 
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munitions.  However,  after  several  unsuccessful  flight  tests,  the  lead 
contractor  for  TSSAM  initiated  a  reliability  improvement  program  to 
address  higher  reliability  requirements,  but  demonstration  of  whether 
problems  had  been  resolved  would  have  taken  several  years  and  cost  more 
than  $300  million.  As  costs  for  TSSAM  increased,  the  Army  ended  its 
participation  in  the  program  and  after  a  period  of  declining  budgets  and 
changes  to  threat  scenarios,  a  cost  and  operational  effectiveness  analysis 
was  completed,  which  showed  that  other  options  might  be  adequate  to 
meet  national  security  requirements.  In  2004,  the  Navy  left  the  JASSM 
program  citing  it  as  a  redundant  capability  to  other  systems  in  its 
inventory.  JASSM  was  expected  to  require  minimal  maintenance  while  in 
storage  and  life-cycle  cost  was  to  be  controlled  through  improved 
reliability  and  supportability  achieved  during  development. 


Figure  1 :  JASSM  Missile  Reliability  Assessment  Flight  Test 


Source:  JASSM  program  office. 


Page  3 


GAO-11-112  Defense  Acquisitions 


To  execute  the  acquisition  strategy  and  meet  cost  and  schedule  goals,  the 
Air  Force  used  Total  System  Performance  Responsibility  (TSPR).4  TSPR 
generally  gives  the  contractor  total  responsibility  for  the  entire  weapon 
system  and  for  meeting  DOD  requirements,  with  minimum  government 
oversight.  The  Air  Force  made  initial  JASSM  requirements  flexible  to  allow 
Lockheed  Martin  to  have  clear  control  of  the  design  and  product  baseline. 
Program  officials  stated  this  strategy  was  based  on  other  successful 
programs,  such  as  the  Joint  Direct  Attack  Munition  program,  and  would 
allow  the  contractor  flexibility  to  make  changes  to  meet  cost  and  schedule 
deadlines  without  having  to  consult  with  the  government.  An  example  of 
this  flexibility  was  the  mission  missile  effectiveness  requirement.  The 
effectiveness  requirement  is  the  minimum  number  of  missiles  required  to 
kill  specified  targets  and  was  named  as  a  key  performance  parameter, 
allowing  trades  between  reliability,  survivability,  and  lethality.  In  other 
words,  if  the  program  was  successful  at  achieving  high  levels  of 
survivability  and  lethality,  reliability  could  remain  low,  even  fluctuate,  and 
still  meet  the  stated  parameters.  Quantities  for  JASSM  were  established  by 
reviewing  the  threshold  targets  and  determining  the  number  of  missiles 
necessary  to  meet  operational  damage  criteria,  based  on  missile 
performance  using  the  effectiveness  requirement.  Therefore,  changes  to 
reliability  would  affect  the  quantities  necessary  to  meet  requirements. 


4  In  May  2003,  a  report  of  the  Defense  Science  Board/Air  Force  Scientific  Advisory  Board 
Joint  Task  Force  stated  that  the  TSPR  policy  marginalized  the  government  program 
management  role  and  replaced  traditional  government  “oversight”  with  “insight.”  It  further 
stated  that  the  authority  of  program  managers  and  other  working-level  acquisition  officials 
subsequently  eroded  to  the  point  where  it  reduced  their  ability  to  succeed  on  development 
programs.  We  have  also  found  that  the  use  of  TSPR  in  government  oversight  and 
involvement  led  to  major  reductions  in  various  government  capabilities,  including  cost¬ 
estimating  and  systems-engineering  staff.  The  loss  of  cost-estimating  and  systems- 
engineering  staff  in  turn  led  to  a  lack  of  technical  data  needed  to  develop  sound  cost 
estimates.  GAO,  Space  Acquisitions:  Actions  Needed  to  Expand  and  Sustain  Use  of  Best 
Practices,  GAO-07-730T  (Washington,  D.C.:  Apr.  19,  2007). 
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Figure  2:  Initial  JASSM  Program  Schedule 


1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009 


Source:  GAO  analysis  of  DOD  data. 

Note:  Fiscal  years. 

As  part  of  the  program’s  1995  acquisition  strategy,  the  Air  Force  received 
five  proposals  for  JASSM  and  in  1996  selected  Lockheed  Martin  and 
McDonnell  Douglas  to  begin  a  24-month  risk-reduction  phase.  Following 
the  risk-reduction  phase,  the  Air  Force  planned  32  months  for 
development  and  a  total  of  56  months  from  program  start  to  full-rate 
production  in  2001.  The  program  planned  for  concurrent  developmental 
and  operational  testing  and  evaluation  with  four  flight  tests  planned  before 
initial  production.  The  Air  Force  planned  to  have  nine  fixed-price 
production  lots  from  2001  through  2009  totaling  2,400  baseline  missiles 
with  an  initial  program  cost  estimate  of  $2.2  billion  (fiscal  year  2010 
dollars). 

A  former  Air  Force  official  who  was  an  early  JASSM  program  manager 
stated  the  Air  Force  accepted  Lockheed  Martin’s  proposal,  which  included 
favorable  fixed-price  contract  prices  for  production  lots  1  through  5  with 
the  understanding  that  the  prices  would  increase  after  Lot  5.  JASSM’s 
acquisition  strategy  planned  for  a  74  percent  unit  cost  increase  between 
Lots  5  and  6.  The  cost  increase  between  Lots  5  and  6  was  to  occur  at  a 
time  when  quantities  were  increasing.  Despite  this  planned  cost  increase, 
the  production  unit  costs  would  have  remained  within  the  Air  Force’s 
acceptable  range  established  before  the  competition  and  at  much  less  cost 
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than  TSSAM.  Further,  the  prices  offered  by  Lockheed  Martin  for  the  first 
five  production  lots  were  below  the  Air  Force’s  desired  cost  range  for  the 
system.  Air  Force  officials  said  the  low  costs  contributed  to  Lockheed 
Martin’s  selection.  However,  to  maintain  the  benefits  of  this  pricing,  the 
quantities  purchased  by  the  Air  Force  had  to  remain  within  a  certain  range 
for  each  of  the  first  5  years. 


Figure  3:  JASSM’s  Original  Acquisition  Strategy 


Notes:  The  increase  between  Lot  5  and  Lot  6  represents  a  74  percent  increase  in  unit  costs.  Fiscal 
year  2010  dollars.  Quantities  are  all  Air  Force. 

While  the  Air  Force  planned  for  some  cost  growth  in  the  original 
acquisition  strategy,  the  program’s  cost  grew  much  more  than  expected. 
For  the  first  four  production  lots,  the  Air  Force  benefited  from  the 
favorable  prices  in  the  original  contract.  However,  because  of  funding 
limitations,  it  was  not  able  to  procure  the  minimum  missile  purchase  in 
Lot  5  and  had  to  renegotiate  this  lot  with  Lockheed  Martin.  In  doing  so, 
Lockheed  Martin  was  able  to  renegotiate  Lot  5  prices  based  on  its  actual 
production  costs — at  over  $1  million  per  missile.  Air  Force  documentation 
indicates  that  previously  negotiated  unit  prices  for  Lot  1  through  Lot  5 
were  as  much  as  45  percent  less  than  Lockheed  Martin’s  actual  costs. 
Subsequent  lots  that  had  not  been  negotiated  under  the  original  contract 
similarly  reflected  an  increase  in  price. 
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Most  of  this  cost  growth  took  place  prior  to  2006,  culminating  in  a  critical 
Nunn-McCurdy  unit  cost  breach  late  in  2006.  According  to  program 
documents,  several  causes  have  been  cited  for  the  critical  Nunn-McCurdy 
unit  cost  breach:  an  unrealistic  cost  estimate  resulting  from  a  flawed 
acquisition  strategy;  the  addition  of  2,600  more  expensive  JASSM-ER 
variants;  the  costly  efforts  to  overcome  reliability  problems;  and  reduced 
annual  production  rates  for  a  longer  period.  Following  the  critical  Nunn- 
McCurdy  unit  cost  breach,  the  Air  Force  halted  production  of  the  missiles 
until  DOD  certified  that  the  program  should  continue.  The  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
(USD/ATL)  found  no  lower  cost  alternatives  and  certified  the  program  in 
2008,  despite  the  missile’s  higher  than  projected  production  costs. 

Since  operational  testing  began  in  2001,  the  reliability  of  the  JASSM 
missile  has  been  inconsistent.  The  Air  Force  flight-tested  62  baseline 
missiles  from  January  2001  through  May  2007,  resulting  in  25  failures  and  3 
“no  tests,”  which  was  a  58  percent  reliability  success  rate.  However, 
because  the  program’s  strategy  allowed  for  the  contractor  to  manage  to 
mission  effectiveness  by  combining  reliability  with  other  factors,  the  58 
percent  reliability  rate  was  sufficient  to  meet  mission  effectiveness 
criteria.  The  Air  Force  tracked  reasons  for  flight  test  failures,  but  was  not 
part  of  the  failure  review  boards  until  production  Lot  5,  5  years  after  the 
start  of  operational  testing.  Air  Force  officials  stated  that  until  2006, 
Lockheed  Martin  handled  all  flight  test  failure  review  determinations  and 
made  corrective  actions  internally  and  the  government  was  not  heavily 
involved.  During  the  Nunn-McCurdy  certification  process,  USD/ATL 
directed  the  JASSM  program  to  develop  a  reliability  growth  plan  that 
would  achieve  90  percent  reliability  for  the  baseline  missile.  The  program 
set  a  goal  of  achieving  this  reliability  rate  by  Lot  11,  or  fiscal  year  2013.  In 
addition,  the  JASSM-ER  program  set  a  reliability  goal  of  85  percent  by  Lot 
4,  or  fiscal  year  2014. 

In  our  2000  report  on  JASSM,  we  recommended  the  Secretary  of  Defense 
revise  its  acquisition  strategy  for  the  JASSM  program  to  be  more  closely 
linked  to  demonstrating  that  the  missile  design  is  stable  and  can  meet 
performance  requirements  before  making  the  production  decision.5  DOD 
partially  concurred  with  our  recommendation  stating  that  its  acquisition 
strategy  is  directly  linked  to  knowledge  points,  that  it  is  linked  to  specific 


5  GAO,  Defense  Acquisitions:  Need  to  Revise  Acquisition  Strategy  to  Reduce  Risk  for 
Joint  Air-to-SurJ ace  Standoff  Missile,  GAO/NSIAD-OO-75  (Washington,  D.C.:  Apr.  26,  2000). 
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criteria  established  for  making  the  low-rate  initial  production  decision, 
and  that  the  contractor  is  required  to  meet  these  criteria.  We  concluded 
that,  while  the  Air  Force  had  taken  steps  to  link  production  decisions  for 
JASSM  to  knowledge,  we  did  not  believe  that  the  specific  criteria 
established  to  support  a  production  decision  were  sufficient  to  minimize 
cost  and  schedule  risk. 

In  June  2010,  the  Secretary  of  Defense  announced  an  initiative  to  restore 
affordability  and  productivity  in  defense  spending.  He  stated  that  there  is  a 
need  to  abandon  inefficient  practices  accumulated  in  a  period  of  budget 
growth  and  learn  to  manage  defense  dollars  in  a  manner  that  is  “respectful 
of  the  American  taxpayer  at  a  time  of  economic  and  fiscal  stress.”  He  set  a 
goal  to  save  $100  billion  over  the  course  of  the  5  year  defense  planning 
period.  Subsequently,  USD/ATL  has  issued  guidance  on  delivering  better 
value  to  the  taxpayer  and  improving  the  way  DOD  does  business.  That 
guidance  indicated  that  budget  savings  could  be  found  by  eliminating 
unneeded  and  costly  programs  and  activities  as  well  as  by  conducting 
needed  programs  and  activities  more  efficiently,  such  as  by  stabilizing 
production  rates.  Subsequently,  in  a  September  14,  2010,  memorandum, 
USD/ATL  provided  specific  guidance  to  acquisition  professionals  to 
achieve  this  mandate.  That  guidance  included  23  principle  actions  to 
improve  efficiency,  including  “Mandate  affordability  as  a  requirement”  and 
“Drive  productivity  growth  through  Will  Cost/Should  Cost  management.” 


Corrective  Actions 
Have  Led  to  Improved 
JASSM  Test  Results 
but  Operational 
Effectiveness  Is  Still 
to  Be  Demonstrated 


Since  2007,  the  Air  Force  has  enhanced  its  oversight  of  the  JASSM 
program  and  made  significant  investments  to  improve  its  reliability  as 
directed  by  USD/ATL.  As  a  result  of  increased  reliability  testing  and 
investments  in  reliability  initiatives,  the  Air  Force  has  identified  many  of 
the  root  causes  for  flight  test  failures.  Since  then,  design  changes  and 
other  corrective  actions  have  improved  JASSM  baseline’s  test  results 
significantly — now  demonstrating  85  percent  success.  The  JASSM-ER 
variant  has  done  well  thus  far,  with  no  scored  failures  during  the  first 
seven  flight  tests.  However,  while  JASSM  baseline  missile  reliability  has 
improved,  it  is  not  expected  to  achieve  the  USD/ATL-required  level  of  90 
percent  until  2013,  and  its  operational  effectiveness  has  not  yet  been 
demonstrated  either  through  operational  testing  or  use  in  a  combat 
operation. 
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Enhanced  Oversight  and 
Increased  Investments 
Have  Led  to  Corrective 
Actions  and  Improved 
JASSM  Reliability 


In  2004,  after  two  back-to-back  flight  test  failures,  the  Air  Force  formed  a 
reliability  enhancement  team  to  address  what  it  considered  the  loss  of 
confidence  in  JASSM’s  performance,  OSD’s  concerns  about  the  program, 
and  budget  reductions.  The  team’s  report  stated  that  while  JASSM’s 
development  and  reliability  were  within  acceptable  ranges  when 
compared  to  other  cruise  missiles,  the  JASSM  program  should  increase 
testing  to  discover  additional  weaknesses  in  design  or  production  as  well 
as  increase  confidence  in  the  level  of  reliability  achieved  and  tie  those 
results  to  contractor  incentives.  In  2007,  after  direction  from  USD/ATL  and 
the  Air  Force  during  the  Nunn-McCurdy  certification  process,  the  program 
office  updated  the  Joint  Reliability  and  Maintainability  Evaluation  Team 
and  Test  Data  Scoring  Board  charters  to  significantly  expand  their  role  in 
management  of  system  development,  manufacturing,  configuration 
changes,  and  testing.  Since  2007,  as  a  result  of  the  Air  Force’s  increased 
attention  to  reliability  testing  and  investments  in  reliability  initiatives,  the 
program  has  identified  many  of  the  root  causes  for  reliability  failures. 
While  there  is  no  single  cause  behind  JASSM  flight  test  failures,  common 
failures  occurred  across  JASSM  subsystems  including  navigation,  flight 
control,  and  propulsion. 
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Figure  4:  JASSM  Subsystems  Involved  in  Flight  Test  Failures 


Notes:  Other  includes  airplane,  power,  and  missile  control  unit.  Unverified  is  due  to  ongoing 
investigations. 


Most  of  the  corrective  actions  to  address  the  causes  of  the  flight  test 
failures  affect  missile  hardware  and  many  have  been  implemented  in  the 
current  configuration  for  new  production  missiles.  However,  some  flight 
test  failure  investigations  are  still  ongoing.  Those  investigations  are  often 
difficult  because  of  the  lack  of  physical  evidence  after  the  flight  test 
missile  detonates  on  the  White  Sands  missile  range.  As  a  result,  identifying 
the  root  causes  for  failures  were  based  on  very  extensive  component 
testing  at  supplier  facilities.  Additionally,  the  root  causes  for  several  test 
failures  were  never  conclusively  determined  as  the  failures  may  have 
resulted  from  aircraft  or  user  malfunctions.  Efforts  to  address  significant 
reliability  problems  found  during  testing  have  contributed  greatly  to 
JASSM’s  cost  growth  and  schedule  delays  since  the  beginning  of 
development.  The  Air  Force  has  estimated  that  it  may  ultimately  spend 
about  $400  million  through  fiscal  year  2025  on  its  reliability  improvement 
initiatives. 

The  Air  Force  has  also  increased  lot  acceptance  testing  of  the  fuses  and 
implemented  high-speed  photography  and  screening  improvements.  In 
addition  to  forensic  evaluation  of  the  missile  impact  area,  the  Air  Force 
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also  employs  visual  inspections  and  a  built-in-test.6  The  Air  Force  has  also 
taken  a  variety  of  actions,  in  addition  to  flight  testing,  to  improve  JASSM’s 
reliability,  including  the  following  initiatives. 

•  Increased  Oversight:  The  Air  Force  and  Lockheed  Martin  have  begun 
a  process  verification  program  to  ensure  suppliers  follow  prime 
contractor  specifications.  According  to  Air  Force  officials,  the  process 
verification  program  has  allowed  the  Air  Force  to  avoid  unforeseen 
costs  as  some  missile  parts  have  become  obsolete.  Further,  officials 
stated  that  it  allows  the  JASSM  program  to  catch  problems  earlier  and 
plan  on  how  to  replace  parts  sooner.  According  to  a  program  official, 
one  process  verification  program  team  caught  an  obsolescence  issue 
with  a  global  positioning  satellite  receiver  and  was  able  to  minimize  the 
cost  and  production  effect  on  the  program. 

•  Missile  Redesign:  Program  officials  state  that  while  wholesale  missile 
redesign  is  not  considered  a  cost-effective  option,  they  are  considering 
design  changes  and  improvements  at  the  component  level. 

•  Increased  Personnel:  The  program  office  has  increased  the  number 
of  government  personnel  supporting  the  process  verification  program 
and  corrective  action  efforts.  During  Lot  1,  the  program  had  two  staff 
members  with  production  and  manufacturing  engineering  expertise — 
by  Lot  7,  22  staff  members  had  such  expertise. 

•  Improved  Quality  Assurance:  In  August  2006,  the  Air  Force  and 
Lockheed  Martin  implemented  a  quality  assurance  program.  Lockheed 
Martin  has  implemented  tests  and  improvement  programs  to  increase 
user  confidence  in  reliability  and  control  costs.  For  example,  Lockheed 
Martin  officials  stated  that,  to  improve  reliability,  they  have  begun 
using  a  test  that  exposes  electrical  connections  to  higher  voltages  than 
they  usually  encounter  during  flight  to  make  sure  the  wiring  can  handle 
a  surge.  Additionally,  Lockheed  Martin  has  increased  the  sample  sizes 
of  certain  components  they  inspect  and  test. 


6  A  built-in-test  provides  fault  detection  to  assess  operational  availability  of  the  missile.  The 
test  is  conducted  on  the  ground  or  via  aircraft  interface  and  evaluates  at  least  95  percent  of 
the  missile  components.  It  does  not  assess  one-shot  devices  such  as  fuse,  wing  retention, 
and  warhead. 
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Recent  JASSM  Testing  Has 
Shown  Improved  Results 


Recent  tests  of  JASSM  have  demonstrated  increased  reliability.  Since  the 
Air  Force’s  reliability  initiatives  began  in  fiscal  year  2007,  the  JASSM 
program  has  conducted  48  missile  flight  tests  and  39  have  been  successful 
(2  were  characterized  as  “no-test”)  for  a  reliability  rate  of  85  percent. 


Figure  5:  JASSM  Baseline  Missile  Test  Results 

Number  of  tests 
25 


Fiscal  year 

|  |  Failures 

Success 

Source:  GAO  analysis  of  DOD  data. 


The  current  focus  of  JASSM  baseline  testing  has  been  on  improving  the 
reliability  of  the  missile.  In  the  most  recent  tests  of  the  JASSM  baseline 
missiles  produced  in  2008,  15  of  16  flight  tests  were  considered  successful. 
In  the  one  failure,  the  warhead  did  not  detonate  and  the  program  is 
awaiting  fuse  recovery  to  make  a  determination  of  the  root  cause.  The 
JASSM-ER  is  in  developmental  testing.  Developmental  testing  of  JASSM- 
ER  is  primarily  addressing  the  differences  of  JASSM-ER  from  the  baseline 
system  (i.e.,  larger  engine  and  fuel  tanks)  and  will  verify  integration  on  the 
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B-l  aircraft.  All  seven  test  flights  of  JASSM-ER  have  been  successful.7  The 
program  office  is  planning  three  additional  integrated  JASSM-ER  tests  to 
be  flown  before  a  production  decision  is  made. 


Uncertain  If  Improved  In  2007,  after  USD/ATL’s  decision  to  enhance  JASSM  reliability,  the  Air 

Missile  Reliability  Will  Force  and  Lockheed  Martin  agreed  to  focus  on  the  inherent  reliability  of 

Result  in  Improved  JASSM  the  missile  and  not  take  into  account  user  error  or  platform  malfunctions 
Orwntinrml  FTferti verier  (Le->  carrier  aircraft,  aircrew  instrumentation,  range  safety,  etc.).  Whereas 

"  operational  testing  is  designed  to  evaluate  the  ability  of  JASSM  to  execute 

a  mission,  reliability  testing  is  more  narrowly  focused  on  evaluating  the 
missile’s  performance  during  the  mission.  While  mission  failures  were 
counted  against  the  program  during  initial  testing,  more  recent  mission 
test  failures  have  been  declared  “no  tests.”  For  example,  in  early  testing 
when  a  B-52  software  issue  resulted  in  an  aborted  mission  and  it  was 
scored  a  test  failure,  this  event  would  have  been  declared  a  no  test  under 
current  missile  reliability  definitions. 

While  recent  flight  testing  of  the  baseline  missile  has  shown  improved 
missile  reliability,  the  Air  Force  has  not  yet  evaluated  the  operational 
effectiveness  and  suitability  of  the  baseline  JASSM  with  all  corrective 
actions  implemented.8  The  JASSM  program  assesses  operational 
effectiveness  through  operational  testing,  follow-on  testing,  and  the 
weapon  system  evaluation  program  (routine  tests  of  inventory  assets). 
These  flight  test  scenarios  assess  operational  effectiveness  in  realistic 
combat  scenarios  against  targets  by  determining  reliability,  evaluating 
capability  and  limitations  by  identifying  deficiencies,  and  recommending 
corrective  actions.  In  operational  testing,  the  JASSM  baseline  program 
flight  tested  38  missiles  from  June  2002  through  May  2007  resulting  in  19 
failures  and  two  no  tests.  While  these  tests  identified  issues  with  missile 
reliability,  they  also  identified  issues  related  to  the  B-52  aircraft,  aircraft 
software,  and  fuse  issues  which  negatively  affected  the  operational 
effectiveness  of  the  missile.  This  led  to  a  9-month  suspension  of  testing  in 
2004  to  address  these  issues.  The  improved  JASSM  baseline  missile’s 


'  OSD  and  Lockheed  Martin  officials  have  not  officially  scored  the  two  most  recent  tests  of 
JASSM-ER  conducted  in  September  2010.  Contractor  officials  stated  that  one  test  was  fully 
successful  while  the  other  experienced  an  in-flight  anomaly.  Contractor  officials  stated  that 
issues  with  the  electronic  control  unit  for  the  new  engine  are  being  evaluated  as  a  possible 
root  cause  for  the  anomaly. 

8  The  program  office  provided  technical  comments  stating  that  while  operational 
effectiveness  has  not  been  evaluated  since  2007,  testing  has  been  more  operationally 
oriented  since  2008. 
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suitability  was  assessed  by  Air  Force  testers  in  2008  and  it  was 
characterized  as  suitable  and  likely  to  meet  reliability  goals;  however, 
operational  testing  of  the  effectiveness  of  the  improved  missile  has  not  yet 
been  scheduled. 


Several  Areas  of 
Potential  Cost  Risk 
Could  Further 
Increase  JASSM 
Program  Costs 


Current  projections  of  JASSM  costs  have  increased  by  over  7  percent  since 
the  Nunn-McCurdy  certification  in  2008.  When  taking  into  consideration 
the  pre-2008  cost  growth,  which  included  the  cost  of  adding  the  JASSM-ER 
variant,  JASSM  has  grown  from  a  $2.2  billion  to  a  $7.1  billion  program.  In 
addition,  while  it  has  initiated  several  cost  control  measures,  the  Air  Force 
appears  to  have  limited  options  to  reduce  JASSM  costs.  Moreover,  several 
areas  of  risk  could  add  to  those  costs.  First,  the  Air  Force  has  not  been 
able  to  provide  enough  annual  funding  to  support  the  annual  procurement 
levels  used  as  the  basis  for  its  2008  program  cost  estimate.  That  has  led  to 
a  less  efficient  production  process  and  a  longer  production  period  (most 
recently  extended  5  years  to  2025).  Second,  until  the  Air  Force  evaluates 
the  effectiveness  of  the  inventory  JASSM  baseline  missiles  with  corrective 
actions  for  previously  identified  hardware  and  software  issues,9  their 
viability  and  military  utility  is  in  question.  If  inventory  missiles  are  found 
not  to  have  utility,  they  may  need  to  be  replaced.  If  retrofitted  missiles  are 
found  to  be  effective,  the  Air  Force  may  still  have  to  find  additional 
funding  to  complete  the  retrofit  process.  Third,  the  Air  Force  plans  to 
conduct  many  more  flight  tests  to  improve  JASSM  reliability  from  85  to  90 
percent.  Finally,  in  comparing  the  capabilities  and  cost  of  JASSM  to 
several  domestic  and  international  missile  systems  in  2008,  the  Air  Force 
assumed  that  JASSM  would  cost  about  $1  million  per  unit,  which  is  about 
40  percent  less  than  currently  expected. 


9  Currently,  the  Air  Force  has  about  942  JASSM  baseline  missiles  in  its  inventory. 
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JASSM  Costs  Are  Much 
Higher  Than  Originally 
Anticipated 


Compared  to  original  program  estimates,  JASSM’s  currently  projected 
costs  are  much  higher  because  of  (1)  higher  than  anticipated  production 
costs,  (2)  longer  production  period,  (3)  the  addition  of  the  JASSM-ER 
variant,  and  (4)  reliability  improvement  efforts.  Through  fiscal  year  2010, 
about  75  percent  of  the  planned  JASSM  quantities  have  yet  to  be  procured 
and,  as  a  result,  most  of  the  program  costs  have  yet  to  be  incurred. 


Figure  6:  Comparison  of  JASSM’s  Original  and  2009  Total  Acquisition  Funding 
Profiles 

Dollars  (in  millions) 

500 


400 


December  2009  selected  acquisition  report 
December  1998  selected  acquisition  report 

Source:  GAO  analysis  of  DOD  data. 


Notes:  Fiscal  year  201 0  dollars.  The  $7.1  billion  includes  2,500  JASSM-ER  missiles. 

Following  the  critical  Nunn-McCurdy  unit  cost  breach,  the  Air  Force 
halted  production  of  the  missiles  until  USD/ATL  certified  that  the  program 
should  continue.  USD/ATL  found  no  lower  cost  alternatives  and  certified 
the  program  in  2008,  despite  the  missile’s  higher  than  projected  production 
costs.  As  a  part  of  our  review,  we  examined  the  cost  estimates  used  by 
OSD  to  certify  the  program  following  the  critical  Nunn-McCurdy  unit  cost 
breach.  This  estimate  used  the  actual  costs  of  the  missile  since  JASSM  was 
well  into  the  production  phase  at  the  time.  Overall,  the  Air  Force’s  cost 
estimate  substantially  met  our  best  practice  standards  in  our  Cost  Guide. 
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For  a  more  in-depth  discussion  of  our  review  of  this  JASSM  cost  estimate, 
see  appendix  III. 

Since  the  Nunn-McCurdy  certification  in  2008,  the  growth  in  JASSM’s 
projected  program  cost  has  moderated,  rising  about  $500  million  (from 
$6.6  billion  to  $7.1  billion)  through  2025.  Reliability  enhancements  to  the 
JASSM  missile  instituted  in  2007  and  additional  reliability  testing  have 
added  the  majority  of  the  increase  in  program  costs.  These  enhancements 
were  implemented  to  meet  USD/ATL’s  90  percent  reliability  goal  which 
was  set  during  the  Nunn-McCurdy  certification.  Also,  the  Air  Force 
decided  to  lengthen  the  program’s  procurement  schedule  by  another  5 
years,  buying  the  same  number  of  missiles  over  a  longer  time  period.  That 
reduces  the  efficiency  of  the  production  processes  and  adds  inflation  to 
the  cost  estimate.  Currently,  on  a  per  unit  basis,  the  average  procurement 
unit  cost  of  a  JASSM  missile  is  projected  to  be  about  $1.2  million.  JASSM- 
ER  is  expected  to  cost  about  $200,000  more  than  the  average,  about  $1.4 
million  per  unit.10 


10  In  May  2008,  USD/ATL  required  the  program  to  prepare  a  new  acquisition  program 
baseline  within  90  days  that  separates  JASSM  and  JASSM-ER  costs.  The  certification  of  a 
new  acquisition  program  baseline  was  originally  delayed  pending  the  analysis  of  flight  test 
failures  and  recently  delayed  until  the  JASSM-ER  requirements  had  been  validated  by  the 
Joint  Requirements  Oversight  Council.  Program  officials  stated  that  the  acquisition 
program  baseline  will  be  completed  to  support  an  initial  production  decision  for  JASSM-ER 
in  November  2011. 
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Table  1 :  Changes  Since  2008  in  JASSM  Estimated  Cost,  Schedule,  and  Quantities 

through  2025 

May  2008 

December  2009 

Percentage 

(Nunn-McCurdy 

(selected 

Change 

certification) 

acquisition  report) 

(2008  to  2009) 

Expected  quantities 

Development  quantities 

106 

118 

11.3 

Procurement  quantities 

4,900 

4,900 

0.0 

Total  quantities 

5,006 

5,018 

0.2 

Cost  estimates 
(dollars  in  millions) 

Development 

$1,383.5 

$1,447.8 

4.6 

Procurement 

$5,257.9 

$5,681.6 

8.1 

Total  program 
acquisition 

$6,641.4 

$7,129.4 

7.3 

Unit  cost  estimates 
(dollars  in  thousands) 

Program  acquisition 

$1,327 

$1,421 

7.1 

Average  procurement 

$1,073 

$1,160 

8.1 

Estimated  delivery  dates 

Production  length 
(years) 

20 

25 

25.0 

Source:  GAO  analysis  of  DOD  data. 

Note:  Fiscal  year  2010  dollars. 


Since  2008,  the  Air  Force  has  added  several  measures  to  control  costs  in 
the  JASSM  program,  but  the  effect  of  these  measures  is  not  yet  clear. 
Examples  of  these  measures  include  the  following. 

•  Contract  Incentives:  The  Air  Force  has  begun  using  fixed-price 

incentive  (firm  target)  contracts  for  each  lot  to  produce  the  missiles  for 
less  than  projected.11 


11  Under  this  type  of  contract,  there  is  a  target  cost,  a  target  profit,  and  a  price  ceiling.  After 
performance,  the  government  and  the  contractor  negotiate  the  final  cost.  If  the  final 
negotiated  cost  is  over  the  target  cost,  then  the  contractor  may  realize  a  profit  less  than  the 
target  profit,  or  even  a  net  loss.  If  the  negotiated  final  cost  is  less  than  the  target  cost,  then 
the  contractor  may  realize  a  profit  greater  than  the  target  profit.  If  the  final  cost  exceeds 
the  price  ceiling,  the  contractor  absorbs  the  difference. 
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•  Greater  Insight  into  Actual  Costs:  Air  Force  officials  have 
increased  insight  into  Lockheed  Martin’s  actual  costs,  which  may  make 
them  more  informed  when  negotiating  new  contracts.  Program  officials 
stated  that,  for  example,  they  now  know  how  many  engineers  are 
needed  to  perform  a  certain  task  and  the  number  of  hours  it  takes  to 
assemble  a  missile.  The  Air  Force  can  directly  verify  the  costs  charged 
by  subcontractors. 

•  Increased  Authority  over  Design:  In  recent  contract  negotiations, 
the  Air  Force  gained  approval  authority  over  certain  design  changes 
that  may  affect  current  and  future  lots,  including  those  that  may 
increase  cost,  require  retrofit,  or  affect  safety.  Previously,  Lockheed 
Martin  had  full  authority  over  most  design  changes. 


While  the  effectiveness  of  the  cost  control  measures  is  not  yet  known,  the 
Air  Force  appears  to  have  limited  options  to  actually  reduce  those  costs. 
For  example,  annual  production  rates  are  expected  to  remain  well  below 
the  levels  projected  at  the  start  of  the  program.  The  2008  Air  Force  cost 
estimate  was  based  on  an  annual  production  rate  of  280  missiles  per  year. 
However,  that  cost  estimate  may  now  be  understated  because  the  program 
has  not  produced  that  many  missiles  in  a  single  year  since  2005.  For 
example,  the  Air  Force’s  procurement  quantities  for  production  Lot  7  and 
Lot  8  were  111  and  80  units,  respectively,  well  below  the  economic  order 
quantity  of  175  missiles  per  year.  Program  officials  stated  that  annual 
quantities  below  the  economic  order  quantity  will  result  in  an  increasingly 
inefficient  production  process  and  some  key  suppliers  may  shift  from 
continuous  to  limited  production.  Further,  Lockheed  Martin  officials 
stated  that  low  production  rates  could  cause  skilled  labor  to  look 
elsewhere  for  work  and  JASSM  reliability  could  be  adversely  affected.  The 
contractor  has  been  able  to  maintain  some  level  of  production  efficiency 
because  of  foreign  military  sales  that  make  up  for  the  reduced  Air  Force 
procurements.  However,  lower  than  projected  annual  procurement  levels 
will  increase  production  costs.  A  further  challenge  is  the  fact  that  JASSM’s 
design  is  mostly  complete  and  there  may  be  few  opportunities  to  reduce 
production  costs  through  redesign.  As  a  result,  average  JASSM  unit  costs 
may  remain  in  excess  of  $1.2  million  indefinitely. 


Effectiveness  of 
Retrofitting  Corrective 
Actions  into  JASSM 
Inventory  Is  Uncertain 


The  Air  Force  has  plans  to  address  the  low  reliability  of  missiles  in  its 
inventory  by  retrofitting  some  of  its  942  missiles  with  hardware  and 
software  corrective  actions.  Program  officials  state  the  retrofit  costs  will 
be  shared  between  Lockheed  Martin  and  the  Air  Force,  but  the  total  cost 
to  retrofit  the  missiles  in  inventory  has  not  been  calculated.  However, 
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previous  efforts  to  retrofit  JASSM  missiles  have  proven  to  be  problematic. 
An  example  of  challenges  associated  with  retrofitting  missiles  is  adding 
telemetry  instrumentation  kits  after  the  missiles  have  been  produced  and 
are  in  the  inventory.  Those  kits  are  added  to  all  missiles  to  be  flight 
tested. 12  This  requires  opening  up  the  missile  to  insert  telemetry  after  the 
stealth  coating  has  been  applied  and  increasing  the  number  of  electrical 
connections  as  compared  to  a  production  missile.13  Air  Force  officials 
stated  the  kit  could  add  some  reliability  concerns  when  it  is  added  to  test 
missiles  because  the  missiles  were  not  designed  to  be  opened  after  they 
were  completed.  Air  Force  officials  also  stated  that  workers  have  to 
reroute  wires  and  remove  the  engine  so  that  the  self-destruct  mechanism 
can  be  installed  and  all  of  this  rework  inside  the  missile  has  the  potential 
to  lead  to  more  errors  and  cause  additional  reliability  issues. 

The  impact  of  retrofitting  missiles  has  become  evident  in  the  weapon 
system  evaluation  program,  which  is  operationally  representative  flight 
testing  run  by  users  of  the  system  and  focuses  on  the  performance  of 
missiles  in  the  inventory.  JASSM’s  performance  in  this  evaluation  program 
has  not  been  good,  with  7  failures  in  12  tests  from  2006  through  2007,  and 
with  at  least  some  of  the  failures  attributable  to  the  retrofit  process.  The 
addition  of  telemetry  kits  has  also  contributed  to  3  no  tests  during  other 
JASSM  flight  testing.  The  Air  Force  has  not  yet  flight  tested  any  of  the 
JASSM  inventory  missiles  that  have  been  retrofitted  with  all  of  the 
corrective  actions  to  address  reliability  issues.  This  type  of  test  would  be 
important  in  determining  the  viability  of  the  current  inventory  of  JASSM 
missiles  and  would  be  a  key  input  in  the  Air  Force  deciding  whether  or  not 
to  retrofit  the  entire  inventory  of  missiles. 


Additional  Flight  Testing  to 
Further  Improve 
Reliability  Will  Be  Costly 


The  Air  Force  plans  more  flight  tests  in  the  next  few  years  of  new 
production  missiles  to  meet  missile  reliability  goals.  For  the  JASSM 
baseline  missile  to  meet  its  reliability  requirement,  the  Air  Force  is 
planning  to  conduct  up  to  48  additional  flight  tests,  at  a  cost  of  about  $120 


12  The  telemetry  kit  is  added  during  the  production  process  for  newly  produced  missiles  to 
be  tested. 

13  Air  Force  officials  state  that  because  of  the  cost  of  the  telemetry  kits,  about  $350,000,  or 
30  percent  to  40  percent  of  the  total  production  unit  cost,  it  is  cost  prohibitive  to  add 
telemetry  kits  to  every  missile  produced.  Therefore,  the  Air  Force  has  decided  against 
changing  the  design  of  the  missile  to  be  more  testable,  and  adds  telemetry  after  fabrication. 
However,  this  requires  additional  retrofitting  if  the  missiles  do  not  have  telemetry  kits 
installed  while  still  on  the  production  line. 
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million.  In  addition,  most  reliability  issues  with  the  baseline  variant  will 
directly  affect  the  progress  of  the  JASSM-ER  variant  as  the  missiles  are  at 
least  70  percent  common  in  hardware  and  95  percent  common  in  software. 
Anything  learned  during  these  flight  tests  about  the  baseline  applies  to 
JASSM-ER.  According  to  the  Air  Force,  as  many  as  20  additional  flight 
tests  may  be  needed  to  fully  demonstrate  JASSM-ER’s  reliability  goal  of  85 
percent. 14 


Table  2:  Flight  Test  Cost  to  Increasing  Reliability 

Variant  of  JASSM 

Additional  flight 
tests  required  to 
reach  reliability 
requirement 

Current  cost  per 
flight  test 

Additional  flight  test 
costs  to  reach 
reliability 
requirement 

Baseline 

48a 

$2.5  million 

$120  million 

Extended  range 

20 

$3.5  million 

$70  million 

Source:  GAO  analysis  of  DOD  data. 


"The  JASSM  baseline  program  can  achieve  its  missile  reliability  goal  with  either  (1)  21  successful 
reliability  test  shots,  or  (2)  3  more  sets  of  reliability  tests  with  16  test  shots  each — with  no  more  than 
two  failures  in  1  set  of  reliability  tests  and  not  more  than  1  failure  in  each  of  the  other  2  sets  of 
reliability  tests. 


The  $190  million  cost  to  achieve  the  final  percentages  of  missile  reliability 
reflects  the  fact  that  problems  or  weaknesses  become  harder  to  find  and 
correct  as  the  more  obvious  issues  are  corrected.  Program  officials  are 
considering  alternative  means  to  meet  user  needs  for  a  more  reliable 
missile  while  reducing  the  cost  of  JASSM  flight  testing. 


Previous  Analysis  of 
Alternatives  Assumed 
Much  Lower  Costs  of 
JASSM  Missile 


As  part  of  the  Nunn-McCurdy  certification  process  in  fiscal  years  2007 
through  2008,  DOD  assessed  whether  there  were  readily  available 
alternatives  that  provided  as  much  or  more  capability  as  JASSM  at  lower 
cost.  DOD  assessed  programs  ranging  from  direct  attack  munitions  to 
intercontinental  range  missiles.  For  the  JASSM  baseline  missile,  all  of 
DOD’s  existing  programs  were  found  to  be  less  effective  in  terms  of 
lethality,  survivability,  or  capacity.  The  Navy’s  Tomahawk  missile  was  the 
closest  alternative  to  meeting  JASSM’s  capability  but  it  is  not  as  lethal  as 


14  OSD  and  Air  Force  officials  were  not  able  to  provide  an  analytical  explanation  for  why 
the  JASSM  baseline  and  JASSM-ER  variants  have  different  reliability  goals — 90  percent  and 
85  percent,  respectively. 
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JASSM.  Also,  the  Tomahawk  is  launched  from  ships  and  not  from  aircraft, 
as  the  Air  Force  plans  to  use  the  capability. 

DOD  also  evaluated  new  or  modified  programs  as  possible  alternatives  to 
JASSM  and  JASSM-ER.  The  Air  Force  evaluated  12  domestic  and 
international  missile  systems  with  projected  unit  production  costs  ranging 
from  $600,000  to  $2.8  million.  JASSM-ER,  estimated  at  the  time  of  this 
evaluation  to  cost  $1  million  per  unit,  was  more  expensive  than  5 
alternative  systems  under  consideration.  In  terms  of  performance,  some 
alternatives  were  more  capable  than  JASSM  and  some  were  not.  All  of  the 
alternative  systems  were  expected  to  require  some  up-front  investment. 
Based  on  this  analysis,  no  other  alternative  was  found  to  provide  greater 
or  equal  military  capability  at  less  cost  than  JASSM-ER. 

Later  in  the  Nunn-McCurdy  process,  however,  OSD  cost  analysts  found 
that  the  costs  of  JASSM-ER  would  likely  be  at  least  $1.4  million  per 
missile.  That  continues  to  be  the  projected  unit  cost  of  JASSM-ER  and,  as 
we  discussed  earlier  in  this  report,  there  are  cost  risks  that  may  drive  that 
unit  cost  higher.  Despite  the  higher  production  unit  costs  for  JASSM-ER, 
the  Air  Force  has  not  revisited  the  results  of  its  assessment  of  alternatives. 
In  light  of  the  current  cost  projections,  which  are  40  percent  higher  than 
assumed  in  the  previous  assessment,  JASSM-ER  would  be  equal  to  the  cost 
of  an  additional  alternative.  Further,  the  unit  cost  differential  between 
JASSM-ER  and  the  lower-priced  alternatives  may  now  be  large  enough  to 
make  those  alternatives  more  competitive  in  terms  of  cost  or  capabilities. 


Conclusions 


DOD’s  25-year  history  to  acquire  and  field  an  affordable  air-to-ground 
cruise  missile  has  been  a  difficult  one.  After  abandoning  the  expensive 
TSSAM  program,  the  Air  Force  conceived  the  JASSM  program  using  an 
acquisition  strategy  that  minimized  government  oversight.  After 
restructuring  the  program  in  2008  and  after  considerable  effort  to  improve 
reliability,  the  JASSM  program  as  it  exists  today  is  much  different  than 
originally  envisioned.  A  $2.2  billion,  11-year  program  to  produce  2,400 
missiles  has  become  a  $7.1  billion,  28-year  program  to  produce  4,900 
missiles.  From  a  technical  and  capability  standpoint,  the  program  offers 
more  now  than  the  baseline  missile  did  before  2008.  Yet,  the  effectiveness 
of  the  new  missiles  remains  to  be  demonstrated  in  operational  testing,  and 
low  production  rates,  retrofit  costs,  and  additional  reliability  testing  could 
drive  program  costs  higher. 

At  this  point,  about  70  percent  of  the  projected  JASSM  costs  have  not  yet 
been  incurred.  In  November  2011,  DOD  will  decide  whether  to  approve  the 
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Air  Force’s  request  to  start  low-rate  initial  production  of  the  JASSM-ER 
variant.  Low-rate  initial  production  is  normally  the  last  major  milestone 
decision  for  an  acquisition  program.  With  the  JASSM  program  now 
expected  to  extend  through  2025  and  about  $5  billion  yet  to  be  spent,  a 
reevaluation  of  its  cost-effectiveness  is  warranted  before  such  a 
commitment  is  made.  This  is  particularly  true  given  the  Secretary  of 
Defense’s  recent  initiative  to  improve  the  cost-efficiency  of  defense 
acquisition  programs.  At  this  juncture,  the  JASSM  program  would  seem  to 
be  an  excellent  opportunity  for  DOD  and  Air  Force  leadership  to  take  a 
hard  look  at  the  cost-effectiveness  and  efficiency  of  this  important  but 
costly  defense  program. 


Recommendation  for 
Executive  Action 


We  recommend  that  the  Secretary  of  Defense  defer  the  production 
decision  for  JASSM-ER  until  (1)  the  program’s  likely  costs  and 
affordability  are  reassessed  to  take  into  account  the  feasibility  and  cost  of 
retrofitting  JASSM  baseline  missiles  or  replacing  them,  the  cost  of 
additional  reliability  testing  against  the  likely  improvement,  and  the  effect 
of  sustained  low  production  rates;  and  (2)  the  results  of  the  previous 
analysis  of  alternatives  are  reassessed  in  light  of  the  likely  costs  of  the 
JASSM  program. 


Agency  Comments 
and  Our  Evaluation 


In  its  comments  on  our  draft  report,  DOD  partially  concurred  with  our 
recommendation.  DOD  stated  that  JASSM-ER  is  on  track  for  a  Milestone  C 
low-rate  initial  production  decision  in  November  2010.  DOD  also  agreed 
that  the  rate  of  JASSM  production  has  not  been  optimum  and  that  it  plans 
to  address  efficient  production  rates  as  part  of  the  JASSM-ER  Milestone  C 
decision.  DOD  also  stated  that  (1)  there  are  no  additional  plans  (nor  is 
there  a  need)  to  retrofit  fielded  JASSMs  above  what  has  already  been 
accomplished  or  is  under  way;  (2)  it  has  revisited  various  alternatives  and 
reaffirms  the  continued  validity  of  its  2008  conclusion  that  none  of  the 
alternative  concepts  provide  comparable  operational  utility  at  or  near  a 
similar  cost  or  schedule  to  JASSM;  and  (3)  in  the  absence  of  viable 
alternatives,  delaying  the  program  further  will  increase  costs  and  further 
postpone  delivering  a  vital  capability  to  the  warfighter.  DOD’s  response  is 
reprinted  in  appendix  II. 

In  concluding  that  retrofits  to  the  inventory  missiles  may  not  be  necessary, 
DOD  does  not  address  the  viability  of  the  current  inventory  of  JASSM 
baseline  missiles  or  the  need  to  replace  some  or  all  of  them.  Until  the  Air 
Force  evaluates  the  effectiveness  of  the  inventory  of  JASSM  baseline 
missiles  with  corrective  actions  for  previously  identified  hardware  and 
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software  issues,  their  viability  and  military  utility  will  still  be  in  question. 
In  addition,  DOD  states  that  the  Air  Force  has  revisited  its  earlier 
assessment  of  alternatives  to  JASSM  and  again  found  that  there  are  none 
with  comparable  utility,  cost,  or  schedule.  This  is  new  information  and 
DOD  did  not  provide  details  for  us  to  assess,  including  whether  the  Air 
Force  factored  in  the  higher  current  projections  of  JASSM  costs.  Finally, 
DOD  did  not  address  the  part  of  our  recommendation  dealing  with  the 
cost  of  additional  reliability  testing  against  the  likely  improvement. 

To  the  extent  DOD  has  made  decisions  on  retrofits  and  reconsideration  of 
alternatives,  these  are  positive  signs,  as  is  its  agreement  to  address  the 
efficiency  of  production  rates.  At  this  point,  it  is  not  clear  whether  the 
reliability  of  the  existing  baseline  inventory  missiles  is  acceptable  or 
whether  additional  reliability  testing  is  warranted.  These  determinations 
are  necessary  to  establish  the  full  value  and  cost  of  the  JASSM  program. 

Beyond  these  steps,  it  is  incumbent  upon  the  department  to  reexamine 
JASSM  before  making  the  production  decision  to  ensure  that  the  program 
is  structured  as  efficiently  as  possible  and  is  still  a  good  investment  given 
the  other  demands  DOD  faces.  DOD’s  agreement  to  address  the  efficiency 
of  JASSM  production  rates  is  a  positive  step.  This  is  particularly  important 
given  the  Secretary’s  current  efficiency  and  affordability  initiative.  DOD 
needs  to  ensure  that  it  has  the  information  available  to  fully  assess  the 
JASSM  investment  before  making  the  production  decision.  If  DOD  needs 
more  time,  then  we  believe  the  decision  could  be  delayed. 

We  also  received  several  technical  comments  from  DOD  and  the  Air  Force 
and  have  made  other  changes  to  our  report. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Defense  and 
interested  congressional  committees.  In  addition,  this  report  will  be  made 
available  at  no  charge  on  the  GAO  Web  site  at  http://www.gao.gov.  If  you 


Page  23 


GAO-11-112  Defense  Acquisitions 


or  your  staff  have  any  questions  about  this  report  or  need  additional 
information,  please  contact  me  at  (202)  512-4841  or  sullivanm@gao.gov. 
Contact  points  for  our  Offices  of  Congressional  Relations  and  Public 
Affairs  may  be  found  on  the  last  page  of  this  report.  GAO  staff  who  made 
major  contributions  to  this  report  are  listed  in  appendix  IV. 


Michael  J.  Sullivan,  Director 
Acquisition  and  Sourcing  Management 
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List  of  Congressional  Committees 

The  Honorable  Joseph  Lieberman 
Chairman 

The  Honorable  John  Thune 
Ranking  Member 
Subcommittee  on  Airland 
Committee  on  Armed  Services 
United  States  Senate 
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To  determine  Joint  Air-to-Surface  Standoff  Missile’s  (JASSM)  current 
production  unit  costs  and  the  extent  they  have  grown,  we  analyzed 
JASSM’s  contracts,  budgets,  and  compared  the  program’s  selected 
acquisition  reports.  We  analyzed  Nunn-McCurdy  documentation  and 
certification  criteria  as  well  as  Air  Force  and  Lockheed  Martin  data  to 
determine  the  causes  for  the  cost  growth  and  critical  breaches.  We 
interviewed  officials  with  the  JASSM  joint  program  office;  Lockheed 
Martin;  the  Office  of  Secretary  of  Defense  Cost  Assessment  and  Program 
Evaluation;  and  a  former  program  official. 

To  determine  the  extent  to  which  Department  of  Defense’s  (DOD)  cost 
estimating  policies  and  guidance  support  the  development  of  high-quality 
cost  estimates,  we  analyzed  the  cost  estimating  practices  of  the  Cost 
Analysis  Improvement  Group  (CAIG),  now  known  as  the  Cost  Analysis 
and  Process  Evaluation  (CAPE),  in  the  development  of  life-cycle  cost 
estimates  for  the  Air  Force’s  Joint  Air-to-Surface  Standoff  Missile  Program 
(baseline  and  JASSM-ER  variants),  against  the  12  best  practices  of  a  high- 
quality  cost  estimate  as  defined  in  our  Cost  Estimating  and  Assessment 
Guide. 1 

We  assessed  each  cost  estimate,  used  in  support  of  the  critical  Nunn- 
McCurdy  unit  cost  breach,  against  these  12  key  practices  associated  with 
four  characteristics  of  a  reliable  estimate.  As  defined  in  the  guide,  these 
four  characteristics  are  comprehensive,  well-documented,  accurate,  and 
credible,  and  the  practices  address,  for  example,  the  methodologies, 
assumptions,  and  source  data  used.  We  also  interviewed  program  officials 
responsible  for  the  cost  estimate  about  the  estimate’s  derivation.  We  then 
characterized  the  extent  to  which  each  of  the  four  characteristics  was  met; 
that  is,  we  rated  each  characteristic  as  being  either  Not  Met,  Minimally 
Met,  Partially  Met,  Substantially  Met,  or  Fully  Met.  To  do  so,  we  scored 
each  of  the  12  individual  key  practices  associated  with  the  four 
characteristics  on  a  scale  of  1-5  (Not  Met  =  1,  Minimally  Met  =  2,  Partially 
Met  =  3,  Substantially  Met  =  4,  and  Fully  Met  =  5),  and  then  averaged  the 
individual  practice  scores  associated  with  a  given  characteristic  to 
determine  the  score  for  that  characteristic. 

To  determine  the  results  of  the  most  recent  tests  and  if  corrective  actions 
have  been  implemented  for  previous  test  failures,  we  analyzed  JASSM 


1  GAO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs,  GAO-09-3SP  (Washington,  D.C.:  March  2009),  p.  71). 
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flight  test  results  and  failure  review  board  findings  to  determine  scoring 
criteria  and  results  to  determine  what  corrective  actions  were 
implemented.  We  determined  whether  recent  flight  test  results  are 
representative  of  the  entire  fleet  by  comparing  and  evaluating  the  lot-by- 
lot  missile  configuration  changes  and  retrofit  activities.  We  interviewed 
officials  with  the  JASSM  joint  program  office;  Lockheed  Martin;  Office  of 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics; 
Director,  Operational  Test  and  Evaluation;  Air  Combat  Command; 
Secretary  of  the  Air  Force  for  Acquisition;  Office  of  Secretary  of  Defense 
Cost  Assessment  and  Program  Evaluation;  Joint  Staff;  Air  Force 
Directorate  of  Test  and  Evaluation;  and  a  former  Air  Force  official  who 
was  an  early  JASSM  program  manager  to  better  understand  the  program’s 
objectives  and  original  acquisition  strategy.  We  discussed  recent  DOD 
reliability  initiatives  with  Office  of  the  Director,  Operational  Test  and 
Evaluation  officials. 

To  determine  what  the  Air  Force  has  done  to  control  and  reduce 
production  costs  while  improving  reliability,  we  examined  JASSM’s 
contracts  to  see  what  provisions  have  been  added  as  well  as  Air  Force  and 
Lockheed  Martin  data.  We  interviewed  officials  with  the  JASSM  joint 
program  office;  Lockheed  Martin;  and  Office  of  Secretary  of  Defense 
(OSD)  test  organizations  to  determine  if  testing  reflects  the  current 
effectiveness  of  the  missile.  We  reviewed  our  prior  work  on  best  practices 
for  a  knowledge-based  approach  for  acquisition  programs  in  determining  if 
the  Air  Force’s  approach  to  beginning  the  JASSM-ER  program  meets  best 
practices.  We  compared  requirements  and  other  documents  to  see  if  the 
JASSM-ER  missile  reflects  lessons  learned  from  the  baseline  variant  as 
well  as  increased  knowledge  and  oversight  from  the  government.  We 
compared  the  baseline  design  with  JASSM-ER  to  determine  commonality. 
We  interviewed  officials  with  the  JASSM  joint  program  office;  Lockheed 
Martin;  Office  of  Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics;  Office  of  Secretary  of  Defense  Cost  Assessment  and 
Program  Evaluation;  Director  of  Operational  Test  and  Evaluation;  Air 
Combat  Command;  Secretary  of  the  Air  Force  for  Acquisition;  Joint  Staff; 
and  a  former  program  official  to  the  determine  acquisition  planning 
leading  up  to  JASSM-ER’s  production  decision  and  what  initiatives  have 
been  taken  to  control  costs. 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 


ACQUISITION, 
TECHNOLOGY 
AND  LOGISTICS 


SEP  3  0 


Mr.  Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Mr.  Sullivan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report, 

GAO-10-573,  “DEFENSE  ACQUISITIONS:  DoD  Needs  to  Reassess  Joint  Cruise  Missile  Costs 
Before  Starting  New  Production  Phase,”  dated  August  25,  2010  (GAO  Code  120868). 

The  DoD  partially  concurs  with  the  recommendation.  The  rationale  for  our  position  is 
enclosed. 

We  appreciate  the  opportunity  to  comment  on  the  draft  report.  My  point  of  contact  for 
this  effort  is  Mr.  Ron  Woods,  Ronald.Woods@osd.mil.  703-697-8183. 

Sincerely, 


David  Q/Ahenr  ' — 

Dlrector 

Portfolio  Systems  Acquisition 


Enclosure: 
As  stated 
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GAO  Draft  Report  Dated  August  25,  2010 
GAO-10-573  (GAO  CODE  120868) 


‘DEFENSE  ACQUISITIONS:  DoD  Needs  to  Reassess  Joint  Cruise  Missile 
Costs  Before  Starting  New  Production  Phase” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RECOMMENDATION  1 :  We  recommend  that  the  Secretary  of  Defense  defer  the 
production  decision  for  JASSM-ER  until  (1)  the  program’s  likely  costs  and  affordability 
are  reassessed  to  take  into  account  the  feasibility  and  cost  of  retrofitting  JASSM  baseline 
missiles  or  replacing  them,  the  cost  of  additional  reliability  testing  against  the  likely 
improvement,  and  the  effect  of  sustained  low  production  rates;  and  (2)  the  results  of  the 
previous  analysis  of  alternatives  are  reassessed  in  light  of  the  likely  costs  of  the  JASSM 
program. 

DOD  RESPONSE:  Partial-concur.  Having  passed  all  qualification  testing  and 
production  readiness  reviews,  JASSM-ER  is  on  track  for  a  Milestone  C  Low  Rate  Initial 
Production  decision  in  November  2010.  JASSM-ER  Flight  test  results  have  met 
expectations. 

The  Department  agrees  with  GAO’s  assessment  that  the  rate  of  JASSM  production  has 
not  been  optimum.  Lower  than  optimum  production  rates  in  Lots  5  through  8  were 
driven  by  the  need  to  restructure  the  program  to  address  reliability  concerns  resulting 
from  Weapon  System  Evaluation  Program  failures  in  2006-2007  and  Lot  5  test  results. 
The  Department  accepted  the  near-term  cost  impacts  of  the  production  delays  to  lay  a 
solid  foundation  for  production  of  a  more  reliable  baseline  JASSM  and  future  JASSM- 
ER  low-rate  production.  Baseline  JASSM  has  been  in  production  since  December  2001. 
JASSM-ER  has  demonstrated  readiness  for  a  production  decision.  The  Department  plans 
to  address  efficient  production  rates  as  part  of  the  JASSM-ER  Milestone  C  decision. 

Lot  5  and  6  missiles  have  successfully  been  retrofitted  with  new  fuzes,  cabling  and  other 
corrective  actions  to  increase  their  reliability.  Below  a  minimum  acceptable  level,  the 
prime  contractor  continues  to  bear  responsibility  for  determining,  validating  and 
implementing  all  fixes  to  return  the  missile  to  a  specified  reliability  level.  The  Air  Force 
has  also  accomplished  retrofits  and  additional  screening  on  previous  blocks  of  missiles 
(Lot  1  and  part  of  Lot  2)  since  2007.  There  are  no  additional  plans  (nor  need)  to  retrofit 
currently  fielded  JASSMs  above  what  has  already  been  accomplished  or  is  under  way. 
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The  Air  Force  issued  a  Request  for  Information  (RFI)  on  alternate  cruise  missile  concepts 
and  completed  an  associated  analysis  in  2007-2008.  The  Air  Force  recently  reviewed  this 
analysis  and  affirmed  the  continued  validity  of  the  2008  conclusion  that  none  of  the 
alternative  concepts  provide  comparable  operational  utility  at  or  near  a  similar  cost  or 
schedule.  In  the  absence  of  viable  alternatives  (i.e.,  no  other  concepts  offer  similar 
capability  at  or  near  same  cost),  delaying  the  program  further  will  increase  costs  and 
further  postpone  delivering  a  vital  stand-off  capability  to  the  warfighter. 
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After  reviewing  documentation  submitted  by  the  JASSM  program  office, 
conducting  interviews,  and  reviewing  relevant  sources,  we  determined  the 
CAIG’s  life-cycle  cost  estimate’s  totaling  $7.1  billion  for  both  programs — 
the  JASSM  baseline  cost  estimate  was  $3.4  million  while  the  JASSM-ER 
variant  cost  estimate  of  $3.7  million — Fully  Met  one  and  Substantially  Met 
the  other  three  characteristics  of  a  reliable  cost  estimate,  as  shown  in 
Table  3  below.  We  assessed  12  measures  consistently  applied  by  cost 
estimating  organizations  throughout  the  federal  government  and  industry 
and  considered  best  practices  for  the  development  of  reliable  cost 
estimates.  We  analyzed  the  cost  estimating  practices  used  by  CAIG  in 
developing  the  life-cycle  cost  estimates  for  both  programs  against  these  12 
best  practices  and  the  findings  are  documented  in  table  3  below. 


Table  3:  Extent  to  Which  the  Cost  Estimating  Practices  for  JASSM  Baseline  and  JASSM-ER  Reflect  GAO  Best  Practices 


Best  practice 

Explanation 

Satisfied? 

Comprehensive 

The  cost  estimates  should  include  both  government  and  contractor  costs  of  the  program 
over  its  full  life-cycle,  from  inception  of  the  program  through  design,  development, 
deployment,  and  operation  and  maintenance  to  retirement  of  the  program.  They  should  also 
provide  a  level  of  detail  appropriate  to  ensure  that  cost  elements  are  neither  omitted  nor 
double-counted,  and  they  should  document  all  cost-influencing  ground  rules  and 
assumptions. 

Substantially  Met 

Well  documented 

The  documentation  should  addresses  the  purpose  of  the  estimate,  the  program  background 
and  system  description,  its  schedule,  the  scope  of  the  estimate,  the  ground  rules  and 
assumptions,  all  data  sources,  estimating  methodology  and  rationale,  the  results  of  the  risk 
analysis,  and  a  conclusion  about  whether  the  cost  estimate  is  reasonable.  Therefore,  a  good 
cost  estimate — while  taking  the  form  of  a  single  number — is  supported  by  detailed 
documentation  that  describes  how  it  was  derived  and  how  the  expected  funding  will  be 
spent  in  order  to  achieve  a  given  objective.  Finally,  the  cost  estimate  should  be  reviewed 
and  accepted  by  management  to  ensure  that  there  is  a  high  level  of  confidence  in  the 
estimating  process  and  the  estimate  itself. 

Fully  Met 

Accurate 

The  cost  estimate  should  provide  for  results  that  are  unbiased  and  should  not  be  overly 
conservative  or  optimistic.  Among  other  things,  the  estimate  should  be  grounded  in  a 
historical  record  of  cost  estimating  and  actual  experiences  on  comparable  programs. 
Estimates  are  accurate  when  they  are  based  on  an  assessment  of  most  likely  costs, 
adjusted  properly  for  inflation,  updated  regularly,  and  contain  few,  if  any,  minor  mistakes. 

Substantially  Met 

Credible 

The  cost  estimates  should  discuss  any  limitations  of  the  analysis  because  of  uncertainty  or 
biases  surrounding  data  or  assumptions.  Major  assumptions  should  be  varied,  and  other 
outcomes  recomputed  to  determine  how  sensitive  they  are  to  changes  in  the  assumptions. 
Risk  and  uncertainty  analysis  should  be  performed  to  determine  the  level  of  risk  associated 
with  the  estimate.  Further,  the  estimate’s  results  should  be  crosschecked,  and  an 
independent  cost  estimate  conducted  by  a  group  outside  the  acquiring  organization  should 
be  developed  to  determine  whether  other  estimating  results  produce  similar  results. 

Substantially  Met 

Source:  GAO  analysis  of  DOD  data. 


The  following  explains  the  definitions  we  used  in  assessing  CAIG’s  cost 
estimating  methods  used  in  support  of  the  critical  Nunn-McCurdy  unit  cost 
breach: 
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•  Fully  Met — -JASSM  program  office  provided  complete  evidence  that 
satisfies  the  entire  criterion; 

•  Substantially  Met — -JASSM  program  office  provided  evidence  that 
satisfies  a  large  portion  of  the  criterion; 

•  Partially  Met — -JASSM  program  office  provided  evidence  that  satisfies 
about  half  of  the  criterion; 

•  Minimally  Met — -JASSM  program  office  provided  evidence  that 
satisfies  a  small  portion  of  the  criterion;  and 

•  Not  Met — -JASSM  program  office  provided  no  evidence  that  satisfies 
any  of  the  criterion. 

The  sections  that  follow  highlight  the  key  findings  of  our  assessment. 


JASSM  Baseline  and 
JASSM-ER  Estimates 
Substantially  Met 
Characteristics  for 
Comprehensiveness 


Though  the  cost  estimates  accounted  for  all  possible  costs  and  were 
structured  in  such  a  manner  that  would  ensure  that  cost  elements  were 
omitted  or  double-counted,  neither  the  JASSM  baseline  nor  JASSM-ER  had 
a  Work  Break  Down  Structure  (WBS)  dictionary  that  defined  each 
element.  1  In  addition,  the  JASSM  baseline  variant  provided  no  evidence 
that  risks  associated  with  the  ground  rules  and  assumptions  were  traced 
back  to  specific  cost  elements. 

•  All  applicable  costs  including  government  and  contractor  costs 
were  included  in  the  estimates — The  cost  estimates  included  sunk 
costs  such  as  contractor  program  management,  overhead,  system 
design,  and  development  and  testing.  In  addition,  the  program  office 
outlined  the  cost  estimating  methodology,  basis  of  the  costs,  as  well  as 
development  costs  for  JASSM-ER  and  other  government  costs. 

•  The  cost  estimates’  level  of  detail  ensure  that  no  costs  were 
omitted  or  double-counted — The  cost  estimates  are  based  on  a 
product-oriented  WBS  which  is  in  line  with  best  practices.  For 
example,  the  cost  estimate  is  broken  down  into  various  components 
such  as  the  propulsion,  payload,  airframe,  and  guidance  and  control 
and  also  includes  supporting  cost  elements  such  as  systems 


1 A  work  breakdown  structure  is  tire  cornerstone  of  every  program  because  it  defines  in 
detail  the  work  necessary  to  accomplish  a  program’s  objectives.  WBS  reflects  the 
requirements,  resources,  and  tasks  that  must  be  accomplished  to  develop  a  program. 
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engineering,  program  management,  and  system  test  and  evaluation.  As 
a  result,  all  of  the  system  products  are  visible  at  lower  levels  of  WBS 
providing  us  with  confidence  that  no  costs  were  omitted  or  double- 
counted.  WBS  has  been  updated  as  the  JASSM  baseline  and  JASSM-ER 
programs  have  evolved;  however,  there  is  not  an  accompanying 
dictionary  that  defines  each  element  and  how  it  relates  to  others  in  the 
hierarchy. 

•  Ground  rules  and  assumptions  were  largely  identified  and 
documented — The  JASSM  baseline  cost  estimate  documentation 
included  a  list  of  risk  model  inputs  based  on  WBS  elements.  Although 
WBS  elements  such  as  engineering  support,  subcontractor,  and 
warranty  were  identified,  there  was  no  discussion  of  risk  upon 
assumptions  that  drive  costs  such  as  product  reliability,  sustainability 
of  subcontractors,  or  schedule  variability.  Like  the  JASSM  baseline  cost 
estimate  documentation,  the  JASSM-ER  cost  estimate  documentation 
also  included  a  list  of  ground  rules  and  assumptions;  however,  there 
was  evidence  that  risk  associated  with  the  fuel  tank  assumption  was 
traceable  to  a  specific  WBS  element.  In  separate  documentation,  we 
were  able  to  identify  where  the  program  office  considered  risks  for  the 
JASSM  baseline  estimate. 


JASSM  Baseline  and 
JASSM-ER  Estimates  Fully 
Met  Characteristics  for 


•  The  cost  estimate  is  fully  documented — For  the  JASSM  baseline 
and  JASSM-ER,  the  cost  estimate  documentation  included  a  report 
documentation  page  identifying  the  report  date,  title,  contract  number, 
report  authors,  and  other  information.  The  documentation  also 
included  a  table  of  contents,  introduction,  purpose,  and  structure  of  the 
document  as  well  as  the  scope  of  the  estimate,  a  list  of  team  members, 
the  cost  methodology,  and  a  system  description.  The  documentation 
discussed  a  risk  and  sensitivity  analysis,  costs  broken  out  by  WBS 
elements  including  data  sources  and  estimating  method  and  rationale, 
and  provided  evidence  that  the  estimates  were  updated  using  actual 
costs.  In  a  separate  briefing,  the  program  office  outlined  the  cost 
estimating  methodology,  basis  of  the  costs,  as  well  as  development 
costs  for  JASSM-ER  and  other  government  costs.  The  program  office 
also  provided  a  copy  of  the  cost  sufficiency  review  of  the  estimate, 
which  included  the  estimate’s  purpose  and  scope,  technical  description 
and  schedule,  ground  rules  and  assumptions,  data  sources  and  analysis, 


Being  Well  Documented 


Both  cost  estimates  were  documented  in  enough  detail  that  would  allow 
an  analyst  unfamiliar  with  the  program  to  recreate  the  estimate  and  get  the 
same  result.  In  addition,  the  briefing  to  management  was  detailed  enough 
to  show  that  the  estimates  were  credible  and  well  documented. 
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and  methodology.  For  both  programs,  the  estimate  documentation  and 
the  cost  analysis  requirements  document  (CARD)  addressed  best 
practices  and  the  12  steps  of  a  high-quality  estimate.  Contingency 
reserves  and  the  associated  level  of  confidence  for  the  risk-adjusted 
cost  estimate  were  also  documented.  Electronic  versions  of  the  cost 
estimates  were  also  provided. 

•  The  estimate  documentation  describes  how  the  estimate  was 
derived — The  point  estimate  was  developed  primarily  using  actual 
costs,  with  a  few  cost  elements  estimated  based  on  learning  curves 
method.* 2  Actual  sunk  costs  for  prior  years  were  presented  and 
remaining  production  lot  costs  were  based  on  a  labor  staffing 
assessment  and  the  latest  contractor  labor  rates.  Cross-checks  were 
performed  and  no  instances  of  double-counting  were  visible.  A 
separate  document  was  provided  that  showed  in  detail  how  the  cost 
estimate  was  developed,  what  data  were  used  to  create  the  cost 
estimate,  and  how  risks  were  quantified  to  determine  a  level  of 
confidence  in  the  cost  estimate. 

•  The  estimates  were  reviewed  and  approved  by  management — 

The  estimates  were  presented  by  OSD  CAIG  to  the  OSD  overarching 
integrated  product  team  for  consideration  as  the  new  acquisition 
program  baseline.  In  November  2009,  the  team  provided  a  detailed 
overview  of  the  JASSM  program  which  addressed  the  major  cost 
growth  factors,  such  as  the  addition  of  the  JASSM-ER  variant, 
reliability  enhancements,  and  the  reduction  in  missile  purchases. 


JASSM  Baseline  and 
JASSM-ER  Variant 
Estimates  Substantially 
Met  Characteristics  for 
Accuracy 


Both  cost  estimates  were  unbiased  and  represented  most  likely  costs.  For 
example,  the  estimates  were  adjusted  to  reflect  risks  and  the  program 
office  also  included  requirements  in  the  new  Lot  8  contract  that  would 
allow  them  to  update  the  cost  estimates  with  actual  data. 

•  The  cost  estimates  were  adjusted  for  inflation — The  JASSM 
program  office  used  the  February  2009  version  of  the  OSD  inflation 
rates  provided  by  the  Secretary  of  the  Air  Force/Financial  Management 


2  Learning  curve  theory  is  based  on  the  premise  that  people  and  organizations  learn  to  do 
things  better  and  more  efficiently  when  they  perform  repetitive  tasks.  Learning  curves 
assume  that  as  the  quantity  of  units  to  be  produced  doubles  the  amount  of  effort  declines 
by  a  constant  percentage.  GAO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best 
Practices  for  Developing  and  Managing  Capital  Program  Costs,  GAO-09-3SP 
(Washington,  D.C.:  March  2009),  p.  137. 
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Appendix  III:  JASSM  Baseline  and  JASSM-ER 
Cost  Estimates  Met  Most  Best  Practices,  but 
the  Risk  Analyses  Did  Not  Consider 
Reliability  or  Extending  the  Production 
Schedule 


Cost  and  Economics.  The  estimates  were  developed  and  documented 
in  base  year  1995  dollars  and  inflated  using  the  weighted  rates 
applicable  to  the  appropriations  in  the  estimate.  Base  year  1995  is  the 
program’s  designated  base  year. 

•  The  cost  estimates  included  most  likely  costs — Per  the  Nunn- 
McCurdy  certification  process,  the  CAIG  developed  independent  cost 
estimates  for  the  JASSM  baseline  and  JASSM-ER  development  and 
procurement  costs  as  well  as  future-year  resource  requirements  for  the 
baseline  and  JASSM-ER  variants.  Operating  and  support  costs  as  well 
as  software  costs  were  also  included  in  the  estimates.  The  JASSM 
baseline  life-cycle  cost  estimate  of  $3.4  billion,  which  spans  a  period  of 
time  from  2001  through  2015,  was  estimated  at  the  77  percent 
confidence  level,  while  the  JASSM-ER  life-cycle  cost  estimate  of  $3.7 
billion  spans  the  period  from  2011  through  2025  and  was  estimated  at 
the  73  percent  confidence  level. 

•  The  cost  estimates  have  not  been  updated  to  reflect  current 
costs — Though  the  JASSM  baseline  estimate  dated  April  2008  was 
updated  to  reflect  new  program  changes,  the  CARD  has  not  been 
updated  since  May  2003.  Examples  of  JASSM  baseline  changes  include 
additional  reliability  enhancement  team  improvements  and  additional 
testing,  which  are  not  reflected  in  the  2003  CARD.  However,  when 
comparing  the  JASSM  WBS  dated  January  1999  and  the  Lot  8  contract 
dated  January  2010,  it  is  evident  that  WBS  has  been  updated  as  changes 
have  occurred.  On  the  other  hand,  the  CARD  for  the  JASSM-ER  was 
updated  as  of  August  2009.  Updates  to  the  JASSM-ER  CARD  include  a 
new,  more  powerful  engine  than  the  baseline  variant.  As  part  of  the 
Milestone  C  process,  work  is  currently  under  way  by  CAPE  to  update 
the  JASSM-ER  cost  estimate.  The  program  office  said  that  the  JASSM- 
ER  cost  model  will  include  updated  costs  based  on  the  Lot  8  proposal 
data,  updated  quantity  profiles,  and  January  2010  revised  inflation 
rates.  The  program  office  is  in  the  process  of  updating  labor  rates  and 
overhead  rates  and  is  reexamining  all  component  prices. 

•  The  cost  estimates  were  based  on  historical  costs — The  JASSM 
baseline  and  JASSM-ER  share  70  percent  building  materials  and  95 
percent  software  design;  therefore,  the  WBS  is  virtually  the  same.  As 
such,  the  JASSM  baseline  and  JASSM-ER  share  many  of  the  same  costs. 
As  outlined  in  both  the  JASSM  baseline  and  JASSM-ER  cost  estimate 
documentation,  the  cost  estimates  were  based  on  actual  labor  costs 
from  JASSM  baseline  production  Lot  1  through  Lot  4,  actual  material 
and  subcontractor  costs  from  JASSM  production  Lot  1  through  Lot  4, 
and  estimated  labor  costs  for  JASSM  production  Lot  5  and  Lot  6.  Also, 
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Appendix  III:  JASSM  Baseline  and  JASSM-ER 
Cost  Estimates  Met  Most  Best  Practices,  but 
the  Risk  Analyses  Did  Not  Consider 
Reliability  or  Extending  the  Production 
Schedule 


JASSM  Baseline  and 
JASSM-ER  Estimates 
Substantially  Met  Criteria 
for  Credibility 

•  The  estimates  were  assessed  for  risk  and  uncertainty — Both 
programs  identified  engineering  and  test  support,  subcontractors,  and 
warranty  as  major  risk  elements.  However,  the  analysis  did  not  identify 
reliability  or  an  increase  in  the  production  schedule  as  possible  risk 
factors.  During  the  Nunn-McCurdy  certification  process,  the  DOD’s 
analyses  found  that  the  cost  breach  was  driven  by  four  primary  factors, 
two  of  which  focused  on  reliability.  As  a  result,  the  program  office 
instituted  a  reliability  enhancement  program  directed  to  address 
reliability  concerns.  An  indirect  effect  of  the  enhancement  program 
was  an  increase  in  the  overall  missile  costs.  The  December  2009 
selected  acquisition  report  identified  increases  to  the  missile  hardware 
cost  due  to  reduced  annual  quantities,  missile  production  breaks,  and 
increased  test  requirements  and  reliability  programs.  The  combined 
cost  estimate,  for  the  JASSM  baseline  and  JASSM-ER  variants,  has 
grown  significantly  over  time.  By  not  including  reliability  and  the 
extension  of  the  production  schedule  as  possible  risk  factors,  cost 
growth  could  continue  to  occur  in  future  production  lots.  As  a  result, 
the  programs’  calculated  point  estimate  confidence  level  of  77  percent 
for  baseline  and  73  percent  for  the  JASSM  and  JASSM-ER  variants  may 
be  overstated. 


While  the  cost  estimates  addressed  risk  and  uncertainty  as  well  as 
sensitivity,  the  estimates  failed  to  address  the  risks  regarding  reliability 
and  changes  to  the  production  schedule.  By  not  doing  so,  the  program 
office  may  not  have  a  full  understanding  of  the  future  effects  to  the  overall 
cost  position  of  these  two  programs. 
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Figure  7:  JASSM  Baseline  and  JASSM-ER  Cost  Estimate  Trend  Analysis  1997 
through  2009 
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Source:  GAO  analysis  of  DOD  data. 


Note:  DOD  did  not  issue  selected  acquisition  reports  in  December  for  fiscal  years  2000  and  2008. 

•  The  estimates  were  assessed  for  sensitivity — For  both  the  JASSM 
baseline  and  JASSM-ER  estimates,  key  cost  drivers  were  identified.  The 
cost  estimators  examined  eight  cost  factors  for  the  JASSM  baseline 
estimate  and  13  cost  factors  for  the  JASSM-ER  estimate.  For  the  JASSM 
analysis,  engineering  support,  testing  support,  other  subcontractors, 
and  Teledyne  propulsion  had  the  greatest  impact  on  the  total  variance 
in  the  estimate.  Engineering  support  showed  a  14  percent  impact, 
followed  by  test  support  with  a  13  percent  impact,  other 
subcontractors  with  a  12  percent  impact,  and  Teledyne  propulsion  with 
a  9  percent  impact.  These  four  elements  account  for  68  percent  of  the 
total  cost  before  risk  was  applied.  For  the  JASSM-ER  analysis,  the 
Williams  propulsion,  other  subcontractors,  engineering  support,  and 
testing  support  showed  the  greatest  impact  on  the  total  cost  variance  in 
the  estimate.  The  Williams  propulsion  had  a  15  percent  impact, 
followed  by  a  15  percent  impact  for  other  subcontractors,  an  11 
percent  impact  for  engineering  support,  and  a  9  percent  impact  for  test 
support.  These  four  elements  account  for  76  percent  of  the  total  cost 
before  risk  was  applied. 
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•  The  cost  estimates  were  checked  for  errors — Cross-checks  were 
performed  and  no  instances  of  double-counting  were  visible.  The  Lot  5  and 
Lot  6  estimates  were  compared  back  to  Lot  1  through  Lot  4  for  consistency 
and  reasonableness.  Also,  multiple  row  and  column  summation  cross¬ 
checks  were  performed  to  avoid  duplication  and  omission  errors.  Upon 
review  of  the  electronic  cost  model,  GAO  found  no  instances  of  double¬ 
counting  and  the  spreadsheet  calculations  are  accurate  given  the  input 
parameters  and  assumptions. 

•  The  cost  estimates  were  validated  against  an  independent  cost 
estimate — The  CAIG  estimate  is  the  independent  cost  estimate.  As  part  of 
the  Nunn-McCurdy  certification  process,  the  CAIG  developed  an 
independent  cost  estimate  for  the  development  and  procurement  costs  as 
well  as  future-year  resource  requirements  for  the  baseline  and  JASSM-ER 
variants.  This  new  independent  estimate  was  a  joint  effort  by  the  OSD 
CAIG,  the  program  office,  and  the  Financial  Management  Center  of 
Expertise,  so  there  was  no  other  estimate  for  comparison.  Per  the  Nunn- 
McCurdy  JASSM  certification  package  dated  April  30,  2008,  the  CAIG 
estimate  of  the  acquisition  costs  for  the  restructured  JASSM  program  is 
$7. 1  billion,  which  is  directly  comparable  to  the  $6  billion  estimate 
reported  in  the  quarterly  selected  acquisition  report  dated  December  2007. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 
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white.  Pricing  and  ordering  information  is  posted  on  GAO’s  Web  site, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 

MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 
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